Optical microscopy with phototransformable optical labels

ABSTRACT

Imaging a sample that includes phototransformable optical labels (“PTOLs”) with an optical system having a diffraction-limited resolution volume (DLRV), includes providing activation radiation to the PTOLs to activate a statistical subset of the PTOLs. A density of the PTOLs of the activated subset is less than an inverse of the DLRV. Excitation radiation is provided to the activated subset to excite activated PTOLs. Radiation emitted from the activated and excited PTOLs located at different focal planes of the optical system within the sample is detected with the optical system. The preceding steps are repeated one or more times, each time activating a different statistical subset of the plurality of PTOLs. Three-dimensional locations within the sample are determined, with a sub-diffraction-limited accuracy, of the activated and excited PTOLs based on the radiation emitted from the activated and excited PTOLs that is detected from the different focal planes of the optical system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.15/170,521, filed Jun. 1, 2016, entitled, “OPTICAL MICROSCOPY WITHPHOTOTRANSFORMABLE OPTICAL LABELS,” which, in turn, is a continuationapplication of U.S. patent application Ser. No. 14/064,582, filed Oct.28, 2013, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” now U.S. Pat. No. 9,360,426, which, in turn, is a continuationapplication of U.S. patent application Ser. No. 13/655,102, filed Oct.18, 2012, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” now U.S. Pat. No. 8,599,376, which, in turn, is a continuationapplication of U.S. patent application Ser. No. 12/956,946, filed Nov.30, 2010, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” now U.S. Pat. No. 8,462,336, which, in turn, is a continuationapplication of U.S. patent application Ser. No. 12/645,019, filed Dec.22, 2009, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” now U.S. Pat. No. 7,864,314, which, in turn, is a continuationapplication of U.S. patent application Ser. No. 11/944,274, filed Nov.21, 2007, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” now U.S. Pat. No. 7,782,457, which, in turn, is a continuationapplication of, and claims priority to, International Patent ApplicationNo. PCT/US2006/019887, filed May 23, 2006, entitled, “OPTICAL MICROSCOPYWITH PHOTOTRANSFORMABLE OPTICAL LABELS,” which, in turn, claims priorityto U.S. Provisional Patent Application No. 60/683,337, filed May 23,2005, entitled, “OPTICAL MICROSCOPY WITH PHOTOTRANSFORMABLE OPTICALLABELS,” and to U.S. Provisional Patent Application No. 60/780,968,filed Mar. 10, 2006, entitled, “IMAGING INTRACELLULAR FLUORESCENTPROTEINS AT NEAR-MOLECULAR RESOLUTION,” all of which are incorporated byreference for all purposes.

BACKGROUND

A paper by one of the inventors, E. Betzig, Opt. Lett. 20, 237 (1995),which is incorporated herein by reference for all purposes, described amethod to improve the m-dimensional spatial resolution in the image of asample that includes a dense set of discrete emitters (e.g., fluorescentmolecules) by first isolating each discrete emitter in an(m+n)-dimensional space defined by the m spatial dimensions and nadditional independent optical properties (e.g., excitation or emissionpolarization or wavelength of the illumination light, fluorescencelifetime of the fluorescent molecules, etc.). After isolation, the mspatial coordinates of each emitter can be determined with an accuracydependent upon the signal-to-noise-ratio (SNR) of the imaging apparatus,but generally much better than the original spatial resolution definedby the m-dimensional diffraction limited resolution volume (“DLRV”) ofthe imaging optics. The map of all spatial coordinates determined inthis manner for all emitters then yields a superresolution image of thesample in the m-dimensional position space.

Successful isolation of each emitter by this approach requires a meanvolume per emitter in m+n space that is larger than the(m+n)-dimensional point spread function PSF. Consequently, a highmolecular density of emitters (e.g. fluorescent molecules) in the samplerequires high (m+n)-dimensional resolution by the imaging optics. In the1995 paper by Betzig, it was estimated that emitting molecules havingmolecular density of about 1 molecule per cubic nanometer nm could beisolated with near-field microscopy/spectroscopy at cryogenictemperatures (e.g., 77 K) if the molecules were located in a matrix thatintroduced sufficient inhomogeneous spectral broadening. However, withconventional optical microscopy and the broad molecular spectra thatexist under ambient conditions, the density of most target molecularspecies would be far too high for this approach to be used.

SUMMARY

In a first general aspect, a method includes providing first activationradiation to a sample that includes phototransformable optical labels(“PTOLs”) to activate a first subset of the PTOLs in the sample. Firstexcitation radiation is provided to the first subset of PTOLs in thesample to excite at least some of the activated PTOLs, and radiationemitted from activated and excited PTOLs within the first subset ofPTOLs is detecting with imaging optics. The first activation radiationis controlled such that the mean volume per activated PTOLs in the firstsubset is greater than or approximately equal to a diffraction-limitedresolution volume (“DLRV”) of the imaging optics.

In another general aspect, a method of imaging with an optical systemcharacterized by a diffraction-limited resolution volume is disclosed.In a sample including a plurality of PTOL distributed in the sample witha density greater than an inverse of the diffraction-limited resolutionvolume of the optical system, a first subset of the PTOLs in the sampleare activated, such that the density of PTOLs in the first subset isless than the inverse of the diffraction-limited resolution volume. Aportion of the PTOLs in the first subset of PTOLs is excited, andradiation emitted from the activated and excited PTOLs in the firstsubset of PTOLs with the imaging optics is detected. Locations ofactivated and excited PTOLs in the first subset of PTOLs are determinedwith a sub-diffraction-limited accuracy based on the detected radiationemitted from the activated and excited PTOLs.

In another general aspect, a method includes providing activationradiation to a sample that includes phototransformable optical labelsPTOLs to activate a first subset of the PTOLs in the sample.Deactivation radiation, having a spatially-structured radiation fieldincluding intensity minima, is provided to the sample to transformactivated PTOLs to an unactivated state, such that a second subset ofPTOLs located substantially at the minima of the resetting radiationremain activated, while activated PTOLs exposed to the resettingradiation outside the minima are substantially transformed in anunactivated form. Excitation radiation is provided to the sample toexcite at least a portion of the activated PTOLs in the sample, andradiation emitted from the activated and excited PTOLs is detected withimaging optics. The intensity of the first activation radiation iscontrolled and at least one of the intensity and the spatial structureof the deactivation radiation is controlled such that the mean volumeper activated PTOL in the first subset is greater than or approximatelyequal to DLRV of the imaging optics.

In another general aspect, an apparatus includes a position-sensitivedetector adapted for detecting intensities of radiation as a function ofposition on the detector, an optical system characterized by adiffraction-limited resolution volume and adapted for imaging lightemitted from a plurality of activated and excited phototransformableoptical labels (“PTOLs”) in a sample onto the positionsensitive-detector. The PTOLs are distributed in the sample with adensity greater than an inverse of the diffraction-limited resolutionvolume of the optical system. The apparatus also includes a first lightsource adapted for providing first activation radiation to the sample toactivate a first subset of the PTOLs in the sample, a second lightsource adapted for providing first excitation radiation to the sample toexcite a portion of the PTOLs in the first subset of the PTOLs, and acontroller adapted for controlling the activation radiation provided tothe sample such that a density of PTOLs in the first subset of activatedPTOLs is less than the inverse of the diffraction-limited resolutionvolume.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram of interactions between light andfluorescent dyes and between light and PTOLs.

FIG. 2 is a schematic diagram of an optical imaging system, e.g., amicroscope, that illustrates how a single fluorescent emitter ormultiple ones can create diffraction limited images.

FIG. 3 is a schematic diagram illustrating how a sparse subset ofactivated PTOLs can be imaged and localized to sub-diffractive accuracyin one spatial dimension without interfering emission from neighboringPTOLs. The lower half of FIG. 3 illustrates how a second or subsequentactivation can image a sparse subset of remaining PTOLs which in-turncan also be localized to better than diffraction-limited accuracy.Repeated application of this procedure can resolve many individual PTOLsthat are otherwise too close to resolve by conventional fluorescence.

FIG. 4 is a schematic diagram illustrating how a sparse subset ofactivated PTOLs can be imaged and localized to sub-diffractive accuracyin two spatial dimensions without interfering emission from neighboringPTOLs. The images of sparse diffraction-limited spots are on the leftside of FIG. 4, and the localized centers of the spots are rendered ascorresponding images on the rights side of FIG. 4. An accumulation ofsuch images on the right gives the super resolution images of the lowerright corner.

FIG. 5 is a schematic diagram illustrating how different types ofproteins labeled with different PTOL species can be co-localized and howrelative distances and positions within a DRLV of each of the labeltypes can be extracted. Potential uses are in protein co-localizationtests, or affinity tests or affinity mapping, e.g., for synthetic drugdesign.

FIG. 6 is a schematic diagram of an apparatus that can localize PTOLlocations to better than diffractive resolution even if their spacing isless than a DRLV. The components include the PTOL-labeled sample, anactivation subsystem for the PTOLs, an excitation system for PTOLs, animaging/detection system for the emitted light, and a control system forsequencing these tasks and acquiring the data.

FIG. 7 is a flow chart outlining a process in which PTOLs in sampleiteratively are activated, excited, and emit radiation that is detected.

FIG. 8A is a schematic diagram illustrating the use of widefieldmicroscopy for the detection of radiation emitted by PTOLs near thefocal plane of a lens. FIG. 8B is a schematic diagram illustrating thewidefield detection of radiation emitted by PTOLs over a region largecompared to the depth of focus of a detection lens by translating thesample relative to the lens. FIG. 8C is a schematic diagram illustratingthe use of structured excitation in a widefield system to preferentiallyexcite and then detect the radiation emitted from PTOLs in multipleplanes. FIG. 8D is a schematic diagram illustrating the differentpatterns at the detector of a widefield system arising from PTOLs atdifferent planes.

FIG. 9A is a schematic diagram of an exemplary superresolutionmicroscope showing the subsystem used to deliver excitation andactivation radiation via total internal reflection to the sample. FIG.9B is a schematic diagram of the subsystem used to detect the radiationemitted by PTOLs in the exemplary superresolution microscope of FIG. 9A.

FIG. 10A is a schematic diagram illustrating the use of excitationradiation structured in a plane parallel to the focal plane of a lens inorder to provide improved localization precision for individual PTOLs.FIG. 10B compares detection-based and standing wave excitation-basedpoint spread functions useful for localizing individual PTOLs. FIG. 10Cillustrates the generation of a standing wave at a total internalreflection interface between a sample and a substrate by using twocounter-propagating coherent beams that pass through an imagingobjective.

FIG. 11A is a conventional total internal reflection image of a thinsection through several lysosomes in a cell, made visible byfluorescence from a PTOL-tagged, lysosome-specific transmembraneprotein. FIG. 11B is a superresolution image of the same area of thesame section, obtained by isolation and precise localization ofindividual PTOLs.

FIG. 12A is a conventional total internal reflection image of points ofadhesion of a whole fixed cell to a substrate, made visible byfluorescence from a PTOL-tagged version of the attachment proteinvinculin. FIG. 12B is a superresolution image of the same region of thewhole fixed cell, obtained by isolation and precise localization ofindividual PTOLs.

FIG. 13A is a plot of an activation optical lattice at an activationwavelength for a given PTOL species. FIG. 13B is a plot of an excitationoptical lattice at an excitation wavelength for the given PTOL species.FIG. 13C is an effective overall signal producing lattice based on theoverlap of the activation and excitation lattices in FIGS. 13A and B,respectively. FIG. 13D is a plot of a single intensity maximum withinthe activation lattice in FIG. 13A. FIG. 13E is a plot of a singleintensity maximum within the excitation lattice in FIG. 13B. FIG. 13F isa plot of a single effective overall signal generating region within theoverall signal producing lattice in FIG. 13C.

FIG. 14A is a plot of an activation optical lattice at an activationwavelength for a given PTOL species. FIG. 14B is a plot of adeactivation optical lattice at a deactivation wavelength for the givenPTOL species, consisting of a deactivating intensity shell with acentral node at each lattice point. FIG. 14C is a plot of an excitationlattice at an excitation wavelength for the given PTOL species. FIG. 14Dis an effective overall signal producing depletion lattice based on theoverlap of the activation, deactivation, and excitation lattices inFIGS. 14A-C, respectively. FIG. 14E is a virtual image of a 3D testobject obtained by the depletion lattice in FIG. 14D. FIG. 14F is avirtual image of the same 3D test object obtained by conventionalconfocal microscopy.

FIG. 15 is a schematic diagram of how a sub-diffractive limited latentimage can be rendered using PTOLs. In this example PTOLs are embedded ina chemically amplified resist. Exposure of part of the area of theresist to a patterning beam can release acids in that area. Such acidsin turn can change the optical properties of the neighboring PTOLs.

DETAILED DESCRIPTION

1. Overview

a. Superresolution Via Isolation and Localization of TransformableLabels

The advent of photoactivated or photoswitched optical labels, such as,for example, photoactivated or photoswitched fluorescent proteins(“FPs”), provides a variable control parameter (colloquially, a “knob”)with which to control the density of activated molecules that contributeto the signal that is detected in the imaging apparatus and that is usedto generate an image of the sample that contains the FPs by this processof molecular isolation and localization. Thus, the density of the FPsthat contribute to the signal can be tailored to the PSF of the imagingoptics to provide an image at the necessary low molecular density at anygiven time.

More generally, a sample can include many optical labels transformablefrom an inactive state (wherein the labels do not produce significantdetectable radiation when excited) to an activated state (wherein thelabels can emit radiation when excited) by virtue of the interaction ofthe transformable labels with their environment. With sufficient controlover at least one activating environmental parameter, a controllable,sparse subset of the labels can be activated. These activated labels canthen be excited into excited states, from which they can emitfluorescence radiation that can be imaged by an optical system. Bycontrolling the activation environment and exciting radiation, the meanvolume per activated and excited label that emits radiation can begreater than the DLRV characteristic of the optical system. By detectingradiation from such a sparse subset of emitting labels, the location ofthe activated and excited PTOLs can be determined with superresolutionaccuracy. Then, the activated labels can be deactivated, and anothersubset of transformable labels, statistically likely to be located atdifferent positions within the sample, can be activated by controllingat least one activating environmental parameter, and fluorescence fromthe second subset of activated labels can be imaged, and their locationscan be determined with superresolution accuracy. This process can berepeated to determine the location of more transformable labels withinthe sample with superresolution accuracy. The determined locations ofall the transformable labels from the different images can be combinedto build up a superresolution image of the sample.

In the specific case of the photoactivatable or photoswitchablefluorescent proteins, the labels are transformed with light, andtherefore these labels represent one class of phototransformable opticallabel (“PTOL”). The activating environmental parameter is then anactivation radiation at an activation wavelength that can transform thelabels to an activated state, and at least one of the intensity or theduration of the activation radiation can be controlled to activate onlya sparse subset of these PTOLs within the sample. However, other formsof energy other than electromagnetic or other environmental parametersmight be used to achieve controllable activation of other types oftransformable labels.

b. Enhanced Resolution Via Overlapped Spatially Structured Activationand Excitation

In another example, a sample can include many PTOLs, and a subset ofPTOLs located at controlled locations can be activated when the sampleis illuminated with spatially-structured activation radiation. Theactivated PTOLs then can be excited with spatially-structured excitingradiation. The overlap of the structure of the activation radiation withthe structure of the exciting radiation is controlled, such that atleast one overlap region of fluorescing PTOLs comparable to or smallerthan the DLRV can be produced. Fluorescence from the subset of theactivated and excited PTOLs then can be detected and recorded. Theactivated PTOLs then can be deactivated, and a second subset of PTOLscan be activated with spatially-structured activation radiation andexcited with spatially-structured excitation radiation, to generate atleast one overlap region of fluorescing PTOLs in a second subset at adifferent location than the first overlap region, and fluorescence fromthe second overlap region can detected and recorded. This process can berepeated at multiple locations in the sample to build up asuperresolution image of the sample.

c. Superresolution Via Spatially Structured Partial Deactivation

In a further example, a sample can include many PTOLs, and the PTOLs canbe activated with spatially-structured activation radiation. Aspatially-structured deactivation radiation field having one or morenodes can then be applied to the activated PTOLs, with nodes of thedeactivation radiation overlapping one or more regions of activatedPTOLs. The deactivation radiation is controlled so that substantiallyall the activated PTOLs are deactivated, except for those activatedPTOLs near each node. Thus, the remaining activated PTOLs are confinedto one or more regions substantially smaller than the DLRV. Theactivated PTOLs that remain after the application of the deactivationradiation lattice can be excited by an exciting radiation field, andfluorescence from the excited PTOLs can be detected and recorded. Theremaining activated PTOLs are then deactivated with another deactivatingfield. This process can be repeated to build up a superresolution imageof the sample.

d. Properties of Phototransformable Optical Labels

FIG. 1 is a schematic diagram illustrating how light interacts withfluorescent dyes and with PTOLs. A fluorescent molecule 101 can bestimulated by excitation radiation 102 from a ground state into anexcited state 103 that emits a portion of the energy of the excitedstate into a fluorescence radiation photon 104. A wavelength of theexcitation radiation can correspond to the energy difference between theground state and the excited state. The molecule 101 then reverts to theground state 105. This cycle of excitation of the molecule 101 byradiation 102 and emission of fluorescence radiation 104 can be repeatedmany times 106, and the fluorescence radiation can be accumulated by amicroscope camera or detector. If there are many such fluorescentmolecules 101 within a diffraction limited resolution volume (“DLRV”) itmight seem difficult to distinguish the fluorescence radiation of onemolecule from another molecule.

In the case of a phototransformable optical label (“PTOL”) molecule oremitter 111, the ability of the PTOL to absorb excitation radiation andtherefore to emit fluorescence radiation can be explicitly turned on byan activating signal, and in certain cases, can be turned off by ade-activating signal. In an inactivated state, a PTOL 111 can be exposedto excitation radiation 112 having a characteristic wavelength, but itwill radiate little, if any, fluorescence radiation at a wavelengthcharacteristic of an activated and excited PTOL. However, when the PTOL121 is irradiated with activation radiation 122, the PTOL 121 can betransformed into an excitable state 123. The activation radiation 122often has a different wavelength than the wavelength of the excitationradiation, but for some PTOLs activation radiation and excitationradiation have the same wavelength and are distinguished by theirintensities. After a PTOL is transformed into an excitable state 123,subsequent illumination of the activated PTOL 123 by excitationradiation 124, which generally has a different wavelength than thewavelength of the activation radiation 122, generally results indetectable emission of fluorescence radiation 126 that has a differentwavelength than the wavelength of the excitation radiation 124. Thisprocess of excitation and emission can be repeated numerous times 128for an activated PTOL 127 until the PTOL eventually bleaches ordeactivates, at which point the PTOL 129 can no longer be excited andcan no longer emit fluorescence radiation.

Thus, a PTOL 121 can be illuminated with activation radiation 122 havingan activation wavelength, thereby transforming the PTOL into anactivated state 123. The activated PTOL 123 can be illuminated withexcitation radiation 124 having an excitation wavelength that isgenerally different from the wavelength of the activation radiation 122to excite the PTOL into an excited state 125, from which the PTOL 125can emit radiation 126 at an emission wavelength that is generallylonger that the wavelength of the excitation wavelength 124. For somespecies of PTOL, the PTOL can be transformed from an activated state 123back to an unactivated state 121, either through spontaneous decay tothe unactivated state or through the application of de-activationradiation.

Several photoactivatable fluorescent proteins useful for superresolutionmicroscopy are described below. An FP is a particular kind ofphototransformable optical label (“PTOL”) or substance whose opticalproperties can be altered by light and that can be used to label aportion of a sample to image optically the portion of the sample. Asused herein “fluorescence” and “fluorescent” generally designate anoptical response of the PTOL. In addition to the common understanding offluorescence (e.g., emission of a photon from a substance in response toexcitation by a more energetic photon) we include other properties thatcan characterize the PTOL. For example, we include emission of a photonin response to multiphoton excitation, or a large elastic optical crosssection that can be activated or deactivated.

One type of PTOL is a variant of the Aequorea victoria photoactivatedgreen fluorescent protein (“PA-GFP”)—a variant of a protein derived fromthe Aequorea genus of jellyfish by genetic modification, as described inG. H. Patterson and J. Lippincott-Schwartz, Science 297, 1873 (2002),which is incorporated herein by reference, for all purposes. Thisvariant can include a isoleucine mutation at the 203 position (T203)(e.g., a histidine substitution at the 203 position) of wild-type GFPand results in a molecule that has a primary absorption peak in itsunactivated state at about 400 nm and a secondary emission peak with anabsorption peak that is about 100× weaker centered around about 490 nm.Radiation is emitted from the excited GFP in a spectrum that centeredapproximately around a wavelength of about 509 nm. After intenseillumination of the PA-GFP with radiation having a wavelength of about400 nm, the 400 nm absorption peak decreases by about 3×, while theabout 490 nm absorption peak increases by about 100×. Therefore,excitation of the PA-GFP with 490 nm excitation radiation to createfluorescence radiation will predominantly show only those PA-GFPmolecules that have been locally activated with prior irradiation withintense 400 nm light. Other forms of photoactivatable GFP can also beused.

Photoswitchable cyan fluorescent protein (“PS-CFP”), as described in D.M. Chudakov, et al., Nature Biotechnol. 22, 1435 (2004), which isincorporated herein by reference for all purposes, has properties thatare similar to those of PA-GFP, except that for PS-CFP, weakillumination with radiation having a wavelength of about 400 nm canyield fairly bright emission at about 470 nm when the PS-CFP is in itsunactivated state, which allows initial set-up and targeting to bereadily performed. Intense excitation of the PS-CFP at the samewavelength, about 400 nm, causes photoswitching of the protein to aversion having a peak in the absorption spectrum of excitation radiationat about 490 nm and an emission peak at about 511 nm. Therefore, imagingPS-CFP labels within a sample by exciting the sample with about 490 nmexcitation radiation and detecting the about 510 nm fluorescenceradiation will predominantly image only those PS-CFP molecules that havebeen activated with prior about 400 nm excitation. PS-CFP emission issomewhat weaker than that for PA-GFP, due to its lower quantum yield,although fluorescence emission at about 510 nm increases by about 300×after activation with the approximately 400 nm radiation, as comparedwith an increase of about 100× at the emission peak for PA-GFP.

Kaede, as described in R. Ando, H. Hama, M. Yamamoto-Hino, H. Mizuno,and A. Miyawaki, Proc. Natl. Acad. Sci. USA 99, 12651 (2002), which isincorporated herein by references for all purposes is like PS-GFP, inthat Kaede shifts its emission band upon activation. However, unlikePS-GFP, activation occurs at a different wavelength (350-400 nm) thanthe peak absorption wavelength in the unactivated state (508 nm), sothat the unactivated protein can be observed at length without causingphotoconversion to the activated state. The fluorescence emissionspectrum in the unactivated state peaks at about 518 nm, while in theactivated state the absorption spectrum (of excitation radiation) andemission spectrum (of fluorescence emission) peak at about 572 nm andabout 582 nm, respectively. Hence, with Kaede, excitation at either theactivated or unactivated peak may cause unintended excitation ofmolecules in the opposite state. An even brighter protein with an evengreater spread in unactivated/activated emission peaks is commerciallyavailable as Kikume Red-Green, and a monomeric type, PA-mRFP1 also hasbeen developed, as described in V. Verkhusha and A. Sorkin, Chemistryand Biology 12, 279 (2005), which is incorporated herein by referencefor all purposes. The long wavelength excitation/emission in theactivated state of these proteins may help reduce background for singlemolecule detection.

Kindling fluorescent proteins (“KFP”), which are described in D. M.Chudakov, et al., Nature Biotechnol. 21, 191 (2003), which isincorporated herein by reference for all purposes have severaldistinguishing characteristics relative to the others FPs describedabove. First, for KFPs, activation occurs at longer wavelengths (525-570nm), which can inflict less damage on a sample and which can be easierto generate. Second, activation under low intensity illuminationnaturally reverses with a half-life of about 50 seconds. Third,activation under low intensity is reversible under illumination withblue light. Fourth, activation under high intensity at 525-570 nm isirreversible, even under illumination in blue light. Thus, molecules notonly can be “turned-on”, but “turned-off” as well, or set permanently“on”. However, KFP1 currently has a relatively low quantum yield and istetrameric.

Dronpa is a bright, monomeric fluorescent protein, described in R. Ando,H. Mizuno, and A. Miyawaki, Science 306, 1370 (2004), which isincorporated herein by references for all purposes that can beactivated/deactivated over many cycles. Activation of Dronpa occurs atabout 400 nm, with the activated molecules having an about 490 nmabsorption peak, and an about 510 nm emission peak. The molecules revertto the unactivated state under continued exposure to the about 490 nmexcitation. This cycle of activation/deactivation can be repeated atleast about 100 times, with only a relatively low loss in totalfluorescence during such cycling. However, during such cyclingobservation of the activated molecules can lead to their deactivation,possibly before the deactivation of the molecules is desired.

Given the diversity of PTOL species with different activation,excitation, and emission wavelengths and time constants, it is possibleto construct separate images for each species of PTOLs. Thus, differentcomponents of a sample can be tagged with distinct labels, and eachlabeled object can then be independently identified in asuper-resolution image that can be constructed as disclosed herein.

It is possible to label specific sample features of interest with PTOLs,such that the PTOLs, and therefore the specific sample features, can beimaged. For PTOLs that can be genetically expressed (e.g., thephotoactivable fluorescent proteins), DNA plasmids can be created andinserted into the cell by transient transfection, so that fluorescentprotein PTOLs are produced fused to specific proteins of interest.Likewise, stable transfections that permanently alter the genetic makeupof a cell line can be created, so that such cells produce fluorescentprotein PTOLs. PTOLs also can be tagged to specific cellular featuresusing immumolabeling techniques, or high-specificity small moleculereceptor-ligand binding systems, such as biotin ligase.

2. Superresolution Via Isolation and Localization of PhototransformableOptical Labels

a. General Concepts

Radiation from molecules or emitters can be used for sub-diffractivelocalization of PTOLs when the radiating molecules or emitters areisolated and spaced further apart from each other than the diffractionlimited length scale of the imaging optics. For example, as shown inFIG. 2, excitation radiation 201 can excite an isolated emitter 202 intoan excited state 203. Outgoing radiation 204 emitted from the excitedemitter 203 can be collected by microscope optics 205 and refocused 206onto a diffraction limited spot 207. This spot profile is shown plottedon the axis of position 208 versus emission intensity 209 in the imageplane 208. The image and object plane are scaled by the magnification M.In the image plane 208, the minimum spatial width of this spot ischaracterized by fundamental limitation of resolution of microscopes andis given by the Abbe criteria Δx≈0.5*λ*M/NA, where λ is the wavelengthof emission radiation 204 and NA is the numerical aperture of theobjective 205. One can use this magnified image of the isolated emitterto localize the emitter to sub-diffractive precision by measuring thedistribution of the emission at a detector such as a CCD camera. Thisdata can then be fit or otherwise processed to find the center of thedetected signal. For example, the emission intensity profile of lightemitted from a PTOL and detected on a detector can be characterized bythe discrete data set, {n_(i),}, where n_(i) are the number of photonsdetected in the i^(th) pixel of the detector located at position x_(i).This data can be fit to a peaked function to determine a location of thePTOL. For example, a Gaussian function,

${n_{i} = {\frac{N}{\sqrt{2\pi}\sigma}e^{- \frac{{({x_{i} - x_{c}})}^{2}}{2\sigma^{2}}}}},$can be used to perform the fit. A least squares fit of the data to thepeaked function, for example, can find a value for the peak centerlocation x_(c). In addition other parameters, such as, for example, thetotal number of photons detected, N, and the peak width, σ, (which canbe generally on the order of Δx) can also be deduced from the fit.Errors in n_(i) can be expressed by a value, δn_(i), and likewise theuncertainty in the center position, x_(c), can be expressed as through aparameter, δx. In particular, when the system noise is limited by photonshot noise statistics (meaning δn_(i)=sqrt(n_(i))) arising from thedetected signal and N is the number of photons detected, then theaccuracy to which this center can be localized is given byδx=Δx/sqrt(N). To the extent that N is much larger than unity, thelocalization accuracy 210 can be significantly better than thediffraction limit 221. The data also can be fit to other functions thanthe Gaussian function to determine a center location and width of theposition of a PTOL.

However, it can be difficult to apply this technique to a set ofcontinuously-emitting fluorescent molecules 212 that are spaced soclosely together that they are within Δ× of each other. In this case,the diffractive spots are highly overlapped, such that fitting of theimage of a molecule to obtain a position of the molecule withsuperresolution accuracy is difficult. Thus, in this situation theresolution limit generally is given by standard Abbe criterion 221, i.e.the width of the diffractive limited spot.

However, by selectively activating and de-activating subsets of PTOLswithin a dense set of PTOLs this localization concept can be used evenwhen the optical labels are closely spaced. As shown in FIG. 3, weakintensity activation radiation 301 can bathe closely spaced PTOLs 302. Asmall, statistically-sampled fraction 303 of all the PTOLs absorbs theactivation radiation and is converted into a state 303 that can beexcited by the excitation radiation 304. The emission radiation 305, 307from this activated and excited subset is focused to a set of isolated,diffraction limited spots 308 whose centers can be localized tosub-diffractive resolution 309 as illustrated previously in FIG. 2.After enough photons are collected to generate sufficiently resolvedimages of the PTOLs that are members of the activated and excitedsubset, the activated PTOLs are either deactivated to return to anactivatable state 302 (as in the case of Dronpa or KFP) or arepermanently photobleached to a dark form 313, effectively removing themfrom the system. Another cycle of weak intensity activation radiation311 is then applied to activate a new subset 316 of the remainingactivatable PTOLs 312. The PTOLs in this second subset in turn can beput into the excited state 317 by excitation 315. The radiated light318, 320 is refocused by the microscope lens 319 onto well-separateddiffractive resolution limited spots 321. Once again, fitting of eachpeak can define the sub-diffractive locations 322 of the PTOLs in thesecond subset. Further cycles will extract sub-diffractive locations ofother PTOLs, such as PTOL image locations 323.

As shown in FIG. 4, multiple sub-diffractive resolution images in twospatial dimensions, x and y, of individual PTOLs in a sample can begenerated, and then the multiple images can be combined to generate asub-diffraction limited resolution image of the sample. Images shown inFIG. 4 were generated from experimental data taken with a system asdescribed herein. An initial image of a few discrete PTOLs emitting at awavelength that is imaged by imaging optics is shown in frame 401. Aftera subset of PTOLs is activated with an activation pulse of radiationhaving an activation wavelength different from the wavelength ofradiation that is imaged, more PTOLs are detected, as shown in frame402. Several such frames are recorded until many of theseinitially-activated PTOLs bleach and can no longer emit, as shown inframe 403. At this point, a new activation pulse can convert a newsubset of PTOLs into an activated state, and this new subset of PTOLscan emit radiation at the imaging wavelength when the newly-activatedPTOLs are excited, which results in the image of frame 404. This cyclecan be repeated to generate several hundred or thousands of such imageframes, which can be considered to represent a 3D data stack 405 of PTOLimages, with the coordinates, x and y, on the horizontal plane and thetime, t, on the vertical axis. Then all these individual image frames inthe data stack can be summed to generate a total image that isequivalent to a long time exposure of a diffraction-limited image from amicroscope, as shown in frame 406.

However, if activated PTOLs are sufficiently sparse in the sample, theraw signal from each activated PTOL (e.g., the intensity of the signalon individual pixels of a CCD detector), as shown in frame 407, can befitted with an approximate point spread function (e.g., a Gaussian) togenerate a smoothed, fitted signal, as shown in frame 408, and thecenter x,y coordinates of each PTOL can be determined. The location ofeach PTOL can then be rendered in a new image as a Gaussian centered atthe measured localization position, having a width defined by theuncertainty to which this location is known. This uncertainty can besignificantly less than the original radius of the original,diffraction-limited PTOL image 407 (typically by an approximate factorof sqrt(N), where N is the number of photons detected to generated theimage of the PTOL). For example, if there were 400 photons in the pixelsof the image spot of a PTOL, the uncertainty of the fitted centrallocation can be 1/20 of the size of the original diffraction limitedimage of that PTOL.

Applying this process to images of all the activated PTOLs in frames401, 402, 403, and 404 leads to the corresponding narrow rendered peaksin frames 410, 411, 412, and 413. The widths of these rendered peaks aregiven by their localization uncertainty. Applied to all activated PTOLsin all frames of the data stack 405, this localization process resultsin a list of coordinates for many PTOLs within the sample.Alternatively, the rendered peaks can be accumulated (e.g., summed) togive a superresolution image 414 of a dense set of PTOLs. The emissionof any activated PTOL may persist over several frames until it isbleached or otherwise deactivated. For such a case, an implementation ofthis accumulation is to identify the coordinates across several framesof what is likely to be a common PTOL. This set of coordinates can beaveraged or otherwise reduced to obtain a single, more accuratelylocalized coordinate vector of that PTOL. A comparison of thediffraction limited image 406 and the superresolution image 414illustrates the higher resolution achievable by this process.

When the sample drifts over the course of acquiring a data stack, imagesof the sample can be acquired while simultaneously tracking fiducialmarkers (e.g., a single quantum dots) added sparsely to the sample, andcorrecting for the drift thereby measured during the post-acquisitionanalysis. Such fiducial-based drift compensation may also facilitate theadaptation of the techniques described herein to commercial microscopes,allowing the investigation of samples at various magnifications andunder a variety of established imaging modalities (e.g., differentialinterference contrast (“DIC”)) in addition to using the techniquesdescribed herein. Quantum dots that are sparsely added to samples asfiducial marks to compensate for sample drift observed during dataacquisition can be isolated and localized in the same manner as singlePTOLs.

This process of serial activation of different isolated PTOL subsetsallows an effective way of localizing the positions of a dense set ofPTOLs, such that superresolution images in 1, 2, or 3 spatial dimensionscan be generated, as described in more detail herein. Furthermore, thisprocess can also be independently repeated for different species ofPTOLs within a sample, which have different activation, excitation,and/or emission wavelengths. Separate or combined superresolution imagescan then be extracted using each PTOL species. The extracted positionalinformation of two or more different PTOLs that label two differentbinding proteins can describe co-localization and relative bindingpositions on a common or closely connected target. This can be usefulfor determining which proteins are related to each other.

An example of how multiple PTOL species can be used to provide molecularbinding (e.g., co-localization) information and molecular structuralinformation is illustrated in FIG. 5. For example, two different PTOLspecies 501 and 502 can label two different molecules proteins 503 and504, for example, when the PTOL species 501 selectively binds to protein503, and the PTOL species 502 selectively binds to protein 504. If thesetwo proteins 503 and 504 bind to each other to form a molecular complex506, then the two PTOLs 501 and 502 can be located at a short distance505 from each other and therefore radiate in close proximity to eachother. The distance 505 between such co-localized molecules (e.g.,proteins 503 and 504) can be less than the size of the molecular complex506. Because the PTOL species can be imaged, and their locationsdetermined, independently with the methods and systems described herein,PTOLs 501 and 502 can be distinguished even when their locations aredetermined to be within the diffraction limit. Indeed, if the distance505 is larger than the localization resolution of the systems describedherein, then the quantitative value of the distance 505 between thePTOLs 501 and 502 can provide additional information about how and wherethese proteins 503 and 504 are bound to each other. Furthermore, thespatial orientation of each the PTOLs 501 and 502 can be deduced by themethods described herein (e.g., by observing the polarization of dipoleradiation emitted from the PTOL), which in turn can also providepositional and orientational data on the relative attachment betweenproteins 503 and 504. In one implementation, radiation emitted fromactivated and excited PTOLs in a sample can be passed through apolarization filter to discriminate the emitted radiation on the basisof the emitted radiation's polarization. Because the polarization ofemitted radiation is indicative of the dipole orientation of the PTOLfrom which the radiation is emitted, the polarization-sensitive signaldetected at the detector provides information about the orientation ofthe emitting PTOL. This method can be extended to a larger multiplicityof various PTOL species 507, 508, 509, and 510. Co-localizationexperiments could determine which PTOL species 507, 508, 509, and 510bind to each other or, for example, to another target 511. Relativedistances 512 between PTOLs 507, 508, 509, and 510 bound to a target 513can be derived from the localization methods described herein and can beused to map the type and position of the binding sites on the target513.

One implementation of these principles of affinity identification andco-localization measurements is in drug discovery. In particular forsynthetic drug design there is interest in mapping where and howstrongly a library of smaller molecules can bind to various parts of thesurface of a target. If a collection of such low affinity fragments canbe identified and tethered together then as a group they will have highaffinity for the target. There are several techniques utilized bycompanies in identifying such drug fragments out of a library. Such astructure activity relationship and proximity sensing can be identifiedby several techniques, for example, NMR, X-ray Crystallography, chemicalligation with mass spectroscopy, or surface plasmon resonance.

A similar approach can identify structural activity relationships usingmultiple PTOL labeled drug fragments and can identify and localize themwith the phototransformable optical localization approaches describedherein. For example, various PTOL species can label a library ofdifferent molecules (e.g., drug fragments) 507, 508, 509, and 510. Theco-localization of a molecule 507, 508, 509, or 510 with a target 514could confirm attachment of drug fragments to the target 514 and map thebinding affinity of the surface of the target 514 using the methodsdescribed herein. The resulting co-localization and positionalinformation along with any dipole information can be used to design asynthetic drug 514 that would have a high binding affinity to a targetsuch as 511.

b. General Hardware and Software Requirements

FIG. 6 is a schematic view of a PTOL microscope. A sample 601 that hasbeen labeled with PTOLs emits radiation that is collected with animaging lens (e.g., a microscope objective lens) 602 and that can befiltered with one or more filters 604. Images of currently activatedPTOLs are formed at detector 606, which in one implementation can detectsingle photons. Optical elements for providing activation radiation tothe sample can include a light source 607, a shutter 608, a lens 609,and a filter 610. The light source 607 (e.g. one or more lasers, lightemitting diodes, or broadband sources) can emit radiation at anactivation wavelength that causes a PTOL to be transformed from aninactivated to an activated state. The light source 607 can be directlymodulated, or modulated via the shutter 608. The shutter 608 can operateto admit or prevent activation radiation from passing from the lightsource 607 to the sample 601. In one implementation, the shutter can bea mechanical shutter that moves to selectively block the beam path. Inanother implementation, the shutter can be a material that can bemodified electronically or acoustically to admit or prevent light frompassing or to alter a beam path from the light source 607. The filter610 can block certain wavelengths of radiation while passing otherwavelengths. For example, if the sample 601 contains several species ofPTOLs, each having different activation wavelengths, the light sourcemay emit light at each of the activation wavelengths but various filters610 can be inserted in the beam path between the light source 607 andthe sample to block some activation wavelengths while passing otherwavelengths, such that only one (or a selected few) species of PTOL isexcited. Radiation from the light source 607 can be deflected by apartial reflector 603 (e.g., a beam splitter, a dichroic mirror, aspotted mirror, or a diffractive structure and directed through theimaging lens 602 onto the sample 603. Similarly, excitation radiationthat causes an activated PTOL to be transformed from a de-excited stateto an excited state can also be passed from an excitation light source611, through a shutter 612, a lens 613, and a filter 614 and off apartial reflector 603 to the sample 601. A controller 615 (e.g., ageneral or special purpose computer or processor) can control parametersof the activation and excitation pulses (e.g., the wavelength,intensity, polarization, and duration of pulses of various radiationbeams that reach the sample 601; and the timing of activation radiationpulses and excitation radiation pulses) during an image acquisitionsequence. Of course, the optical elements 607-614 can be arranged inother configurations. For example, the activation optics 607-610 and/orthe excitation optics 611-614 can be configured, as in the module 616,to direct radiation to the sample 601 from outside of the lens 602, orthe excitation radiation can be directed onto the sample from adifferent partial reflector than the activation radiation, etc.Furthermore, there can be a multiplicity of components so that PTOLs ofa different species can also be imaged either in parallel or in aseparate sequential acquisition. For example, there can be additionalcameras, filters, shutters, activation sources, or excitation sources,of different wavelengths associated with the characteristics ofdifferent PTOL species. Data from images formed at the detector 606 arecommunicated to the controller 615 for storage and processing. Forexample, the controller 615 can include a memory for recording orstoring intensity data as a function of position on the detector fordifferent image frames. The controller 615 can also include a processorfor processing the data (e.g., a general or special purpose computer orprocessor), for example, to fit the data recorded for an image of anindividual PTOL to determine a location of the PTOL to sub-diffractionlimited resolution, or to combine the data about the locations ofmultiple PTOLs that are determined with superresolution accuracy togenerate an image of the sample based on the locations of multiple PTOLsthat have been located with superresolution accuracy.

FIG. 7 is a flow chart of a process 700 for creating an image of asample containing multiple relatively densely-located PTOLs. Anactivation pulse of radiation having an activation wavelength isdirected onto a sample to transform a subset of PTOLs in the sample froman unactivated to an activated state (step 702). Excitation radiation isapplied to activated PTOLs in the sample at the excitation wavelength,and radiation that is emitted from activated and excited PTOLs andincident onto the imaging and detecting optics is acquired and saved(step 703). Images of a set of activated PTOLs can be acquired and savedmultiple times. For example, the controller can require that N images ofa set of activated PTOLs are acquired, such that if N images have notyet been acquired (step 704) image acquisition (step 703) is repeated.The excitation radiation can be applied to the sample continuously orcan be switched off between acquisitions of images.

After N images of the subset of activated PTOLs are acquired, and ifmore images are to be obtained from the sample (step 705) anotheractivation pulse can be applied to the sample to activate another set ofPTOLs (step 702). Excitation radiation can be applied to this other setof activated PTOLs, and radiation emitted from the activated and excitedPTOLs can be acquired and saved (step 703). Multiple sets of PTOLs canbe activated. For example, the controller can require that M sets PTOLsbe activated, such that if M sets have not yet been activated (step 705)another activation pulse is applied (step 703). Thus, the process ofactivating a set of PTOLs, exciting PTOLs within the activated set, andacquiring images from the activated and excited PTOLs can be repeatedmultiple times, for example, until the total pool of available PTOLsbecomes exhausted or until a desired number of images of a desirednumber of different PTOLs within a spatial area or volume is achieved.

While applying the activation and excitation radiation, the number ofiterations N between activation pulses, along with the intensity of theactivation and excitation radiation can be controlled such that the meanvolume per imaged PTOL in an individual image is generally more than theDLRV of the optical imaging system used to detect and localize theindividual PTOLs. The density of activated PTOLs that are capable ofemitting radiation is generally highest in images acquired immediatelyafter the activation pulse and generally decreases as more PTOLsphotobleach during the acquisition of the N image frames. Furthermore,as the process 700 progresses, and the number of activation pulsesincreases from 1 to M, PTOLs within the sample may photobleach, suchthat fewer and fewer PTOLs within the sample are available to beactivated, excited, and imaged. Thus, in one implementation, theintensity and time length of individual activation pulses and theintensity and time length of excitation radiation can be controlled, toreduce the variation in density of activated PTOLs as the processprogresses. For example, using less excitation radiation (possibly withfewer frames N between activation pulses) can reduce the decrease inimaged PTOLs from the first frame after an activation pulse to the Nthframe just preceding the next activation pulse. In another example, theintensity of individual activation pulses can increase as the process700 progresses from the first to the M^(th) activation pulse. This wouldreduce the decrease in the number of imaged PTOLs in the firstacquisition frame after the Mth activation pulse relative to the numberof imaged PTOLs in the first acquisition frame after the firstactivation pulse, thereby compensating for the reduction in the numberof activable PTOLs as the sequence of activation and image acquisitionprogresses. Thus, in the first example, the variation of activated andexcitable PTOLs during an excitation sequence is reduced and in thesecond example the variation of activated and excitable PTOLs during theactivation sequence is reduced. The reduced variation of activated andexcitable PTOLs allows operation, where more PTOLs can be localized perunit time, while not exceeding the density criteria of more than oneimaged PTOL per DLRV.

In one implementation, multiple species of PTOLs within the sample canbe activated, excited, and imaged. For example, steps of applying theactivation pulses (702) and of exciting and imaging (703) can includeapplying pulses of activation radiation and excitation radiation,respectively, having wavelengths corresponding to the differentactivation and excitation wavelengths of different PTOL species. Amultiplicity of detectors and/or filters can also be used in the imagingstep 703 to image different wavelengths of radiation emitted fromdifferent PTOL species. In this manner, multiple independent data setsof images can be acquired. These independent data sets in turn can bereduced to corresponding super-resolution images of each PTOL specieswithin a sample.

c. Exemplary Excitation and Detection Geometries

The process of activating a subset of PTOLs in a sample, exciting someor all of those activated PTOLs, and imaging the activated and excitedPTOLs can be applied in any optical imaging mode, for example, inwidefield microscopy, total internal reflection fluorescence (TIRF)microscopy, confocal microscopy, and multifocal lattice microscopy.

As shown in FIGS. 8a, 8b, 8c, and 8d , widefield microscopy permits manyindividual PTOLs 800 within a sample 810 that reside near the plane offocus 801 of a lens 802 to be localized simultaneously, when the PTOLsare activated at a low enough density that their separations in theplane 801 are generally larger than the diffraction limited 2Dresolution defined by the lens 802. The magnification of the imagingoptics (e.g., including lens 802) is chosen relative to the size ofindividual pixels 803 in a detector 804 (e.g., an electron multiplyingcharge coupled device (EMCCD) camera) that images the PTOLs 800, so thatthe image 805 from each PTOL is dispersed over several pixels tooptimize the localization accuracy for each PTOL. Of course, ifradiation emitted from a particular PTOL were detected by only one pixelit would be difficult to determine the location of the PTOL withsub-diffraction limited accuracy, but if radiation from the PTOL fallson multiple pixels the signals from the different pixels can be fitted,such that the PTOL can be localized with sub-diffraction limitedaccuracy. However, if radiation from a particular PTOL falls on verymany pixels, then it may overlap with the radiation from another PTOL,or the background noise from the greater number of pixels involved maybe increased. In either case, such that the localization accuracy wouldbe relatively low. Thus, a compromise between having an image of a PTOLfall on too many or too few pixels can be obtained.

Widefield microscopy is easily used with the processes described hereinto achieve 2D localization of PTOLs in thin samples (i.e., sampleshaving a thickness comparable to or smaller than the depth of focus 806characterized by the numerical aperture of the lens and the wavelengthof the fluorescence light emitted from the PTOLs). Application to suchthin samples can: a) limit background signal from autofluorescence orunresolved PTOLs in areas away from the focal plane 806 (since suchbackground can degrade the accuracy with which PTOLs are localized); b)reduce the number of potentially photoactivatable molecules within the2D PSF; and c) when the activating energy is delivered through theimaging lens, insure that the PTOLs that are activated are generallywithin the focal plane of the lens, and therefore produce minimallysized spots at the detector and corresponding optimal localization.

One example of such thin sections is the lamellipodial regions ofcultured cells. Another class of thin samples suitable for widefielddetection is thin sections cut from a larger sample using the microtometechniques common to transmission electron microscopy (eithercryosections or sections from resin-embedded cells or tissues). Suchsolid, cut sections insure that the PTOLs remain immobile for accuratelocalization, and permit deeply buried sample features to be imaged,without the problems of out-of-plane autofluorescence, aberrations, andlight scattering that potentially exist when trying to image the samefeatures by widefield microscopy in the original, thicker sample.

As shown in FIG. 8b , in cases where widefield detection of PTOLs can beapplied to samples that are thick compared to the depth of focus of thelens, localization of PTOLs in 3D can be performed by translating thefocal plane along the optical axis 807 of the lens (e.g., by changingthe separation between the lens and the sample) for each activatedsubset of PTOLs that is imaged to create 2D images of multiple planes ofthe sample. These multiple 2D images can be combined digitally to buildan image stack 808 such that a 3D image of each imaged PTOL in thesample is obtained. Then the 3D image of each PTOL can be fitted toobtain a sub-diffraction limited position of the PTOL positions in 3D,by direct analogy to the 2D case described above. A complete 3Dsuperresolution image can be thereby constructed from many subsets oflocalized PTOLs.

Another approach to providing position information for the PTOLs in thedirection defined by the axis 807 of the lens is to apply the excitationlight in a form that is spatially structured primarily along thisdirection, and substantially uniform parallel to the focal plane (sothat the advantage of simultaneous detection in 2D is retained). Thespatially structured field can then be scanned in the axial directionfor each subset of individually resolvable, activated PTOLs, therebypermitting the axial excitation PSF to be measured at each. The knownPSF of the axially structured excitation can then be fit to this data tofind the relative locations of the PTOLs in the axial direction withnanometric precision. The data can then be combined with the localizedcoordinates of the same PTOLs in the focal plane, and further combinedwith similar results from other subsets of activated PTOLs to build adense superresolution 3D image.

As shown in FIG. 8c , such an axially structured excitation field can becreated by impinging the excitation light on the sample 810 in twocoherent beams 811 and 812 from directions that are mirror imaged withrespect to the detection plane. The beams 811 and 812 intersect withinthe sample 810 to produce a standing wave (“SW”) intensity profile 813in the axial direction 807. The beam 811 approaching the sample from thesame side of the focal plane as the lens 802 can pass through the lens,if desired. For samples sufficiently thin such that only a single SWplane 814 of maximum intensity resides within the sample 810, detectionand localization can proceed by axially scanning the maximum intensityplane as described above. For moderately thicker samples, the period 815of the SW, which can be expressed as p=λ sin(θ)/2 (where p is theperiod, λ, is the wavelength of the excitation radiation, and θ is angleeach beam makes with the focal plane) can be increased by decreasing theangle, θ, until only a single, wider SW plane of maximum intensityintersects the sample 810. Alternatively, as shown in FIG. 8d , ifseveral SW maxima reside within the sample 810, PTOLs 800 excited inplanes corresponding to different intensity maxima 816, 817, and 818 canproduce different patterned spots (e.g., spots 819 and 820 from maximum816, spot 821 from maximum 817, and spots 822 and 823 from maximum 818)at the detector due to the differences in 2D detection point spreadfunction that exists in different planes parallel to the plane of focusof the lens 802. For example, an image of a PTOL on the detector due toemission from the PTOL at the focal plane of the imaging optics will besmaller than an image of the PTOL due to emission from the PTOL from aplane that does not correspond to the focal plane. This information canbe used to discriminate from which SW maximum a given PTOL originates.Also, the detected light can be split between M detectors in the casewhere M standing wave maxima reside within the sample, and correctiveoptics (e.g., a phase mask) can be placed between the lens 802 and eachdetector, such that the focal plane for each detector is coincident witha different SW maximum. Those PTOLs in focus at a given detector thencan be localized in either 2D or 3D using the information recorded atthat detector.

A total internal reflection (“TIRF”) geometry also permits simultaneousdetection and 2D localization of multiple photoactivated PTOLs in aplane. In TIRF microscopy, the intensity of excitation radiation thatilluminates the sample exponentially decreases with increasing distancefrom the sample/substrate interface. Because of the exponential decreaseof the excitation radiation as a function of distance from thesample/substrate interface, excitation that is highly localized in the zdirection can be achieved with relatively little autofluorescence,especially when thick specimens are imaged. Also with TIRF microscopy,relatively few PTOLs (both activated and deactivated) are excitedsimultaneously for a given molecular density, so a larger density oftarget molecules can be initially prepared in the sample. Further,evanescent illumination at multiple angles can be used to localize thePTOLs in the z direction as well to a high degree of accuracy.Additionally, the wavelength of activation radiation as well as thewavelength of excitation radiation can be applied via an evanescentfield to further reduce the extent of activated, excited PTOLs in the zdirection.

Excitation radiation and activation radiation for TIRF microscopy can bedelivered to the sample/substrate interface external to the objectivelens using a prism that is optically coupled to the substrate.Alternatively, excitation and activation radiation can be applied to thesample/substrate interface in an epi configuration, with the excitationradiation entering at the rear pupil of the same objective lens that isused to collect fluorescence radiation emitted from PTOLs in the sample,as long as the numerical aperture (“NA”) of the lens yields a maximumillumination angle, θ_(max)>sin⁻¹(NA/n_(sub)), that is greater than thecritical angle for total internal reflection (“TIR”) (where n_(sub) isthe refractive index of the substrate), and the excitation radiationenters the rear pupil in the outer annular region that supports TIR ofthe excitation radiation.

FIGS. 9a and 9b are schematic diagrams of a system that can usethrough-the-objective TIRF excitation radiation to excitesparsely-populated activated PTOLs in a sample, such that radiationemitted from the activated, excited PTOLs can be imaged to producesuperresolution images of the sample via phototransformation, isolation,and localization of multiple subsets of discrete PTOLs within thesample. For continuous excitation of activated PTOLs, light having awavelength of 561 nm emitted from a 10 mW diode-pumped solid-state laser(available from Lasos GmbH, Jena, Germany) is fiber-coupled to anexcitation collimator 900 and provides an excitation input beam 901 thatcan be focused at the rear pupil plane internal to a 60×, 1.45NA totalinternal reflection fluorescence (“TIRF”) oil immersion objective 902(available from Olympus America, Melville, N.Y.). A narrow bandwidthlaser line filter 903 (available from Semrock, Inc., Rochester, N.Y.) isused to reject both emission noise from the laser and autofluorescencegenerated in the optical path prior to the objective 902. For pulsedactivation of the PTOLs, a second diode laser (available form CoherentInc., Santa Clara, Calif.) that can yield about 50 mW of power at anactivation wavelength, λ_(act), of about 405 nm can be fiber-coupledthrough an intermediate galvanometer-based switch (not shown) to anactivation collimator 904 to create a focused activation input beam 905that is similarly filtered by a bandpass filter 906 (available from CVIOptical, Covina, Calif.) before being combined with the excitation inputbeam 901 at a dichroic mirror 907 (available from Semrock, Inc.). Thiscombined input beam 908 then can be reflected from an elliptical spot ona custom-patterned, aluminized mirror 909 (available from Reynard Corp.,San Clemente, Calif.) into the objective 902. The radius, ρ, at whichthe combined beam 908 enters objective 902 can be controlled to be(n_(sample)/NA)*4.35≈4.14 mm≤ρ≤4.35 mm (for n_(sample)≈1.38), such thatthe resulting refracted ray transverses a low autofluorescence immersionoil (e.g., Cargille type FF, available from Structure Probe Inc., WestChester, Pa.) and is incident at the interface between the sample and acover slip 913 (e.g., a #2 thickness cover slip available from FisherScientific, Hampton, N.H.) at greater than the critical angle,θ_(c)≈sin⁻¹(n_(sample)/n_(coverslip)) for which total internalreflection (“TIR”) occurs. An evanescent field can be therebyestablished within the sample, exciting only those molecules within theshort decay length of the evanescent field. A substantial proportion ofthe incident energy of the excitation and activation beams, however, canbe reflected at the interface to yield a combined output beam 910 thatemerges from the objective 902, and that is then reflected from a secondelliptical spot on mirror 909 diagonally opposite the first ellipticalspot on the mirror. This beam 910 is then divided at dichroic mirror 907into an excitation output beam 911 and a separate activation output beam912 that are finally directed to respective beam dumps.

For typical molecular cross-sections (e.g., approximately 10⁻¹⁶ cm²),the reflected excitation beam energy may be 10¹⁵-fold more intense thana PTOL signal beam 914 that emerges from the objective 902, as shown inFIG. 9b . Therefore, a challenge in this through-the-objective TIRFgeometry is the isolation of the molecular signal from both theinterface-reflected excitation beam and any autofluorescence generatedby this beam in the optics encountered thereafter. The mirror 909 aidsin this isolation because the mirror has an elliptical, anti-reflectioncoated, transmissive aperture whose projection perpendicular to theobjective axis matches the 8.7 mm diameter of the rear pupil, andtherefore passes signal beam 914 to the detection optics with highefficiency. Also, for an elliptical reflective spot D times larger thanthe gaussian width of the reflected beam at the spot, only about erfc(D)of the excitation energy is passed onto the detection optics, or ˜2·10⁻⁵to ˜2·10⁻⁸ for D=3 or 4, respectively. Furthermore, since the spotsocclude only a small fraction of the periphery of the rear pupil, theydo not substantially degrade the detection numerical aperture.Consequently, the PSF standard deviation, s, that factors intosub-diffraction limited localization of PTOLs is not substantiallydegraded. Furthermore, the mirror 909 is wavelength insensitive, andtherefore can be used with different excitation lasers and differentPTOLs without replacement. The mirror 909 can include multiple spots tosupport multi-angle, multi-polarization and/or standing wave TIRFexcitation.

After passage through custom spotted mirror 909, the largely collimatedsignal beam 914 emerging from the infinity-corrected objective 902 canbe reflected by a first mirror 915 (as shown in FIG. 9b ) to travelalong the axis of the detection optics. Any remaining excitation light(as well as much of the remaining activation light) travelingsubstantially along this axis can be removed by a Raman edge filter 916(available from Semrock, Inc.). However, because the optical density ofthis filter 916 decreases rapidly with increasing deviation from normalincidence, baffles 917 can be placed on either side of the filter toremove scattered light at higher angles of incidence generated elsewherewithin the system. The filtered signal beam can be focused into afocused beam 918 with an acromatic tube lens 919 (available from EdmundOptics, Barrington, N.J.) onto the face of a back-illuminated,thermoelectrically cooled (e.g., to −50° C.), electron multiplying CCDcamera 920 (available from Andor Scientific, South Windsor, Conn.) tocreate the desired image of isolated single molecules. A 405 nm notchfilter 921 (available from Semrock, Inc.) also can be included tofurther insure that the camera 920 is not saturated when the activationbeam is applied.

To further increase the localization accuracy in the plane of thesample/substrate interface in the TIRF configuration the substrate canbe used as a waveguide to support the propagation of two or moreintersecting excitation beams. These beams then can form a structuredexcitation field within this plane that is evanescent perpendicular tothe interface. For example, as shown in FIG. 10a , two such excitationbeams 1000 and 1001 can create a standing wave (“SW”) intensity profile1002 along one axis 1003 parallel to the interface between the sample1004 and the substrate 1005. Scanning this SW over one period along thisaxis (e.g., at phases, Δ=0° (as illustrated in frame 1006), Δ=120° (asillustrated in frame 1007), and Δ=240° (as illustrated in frame 1008))and capturing images (e.g., as shown in frames 1009, 1010, and 1011) ofthe activated PTOLs at each SW position then can allow the PTOLs to belocalized on the basis of an effective excitation PSF 1012 as shown inFIG. 10b , having a width ˜λ_(exc)/(4n_(sub)) where λ_(exc) is thewavelength of the excitation radiation and n_(sub) is the index ofrefraction of the substrate, which is lower than the detection PSF 1013having a width ˜λ_(ems)/(2NA) present at the CCD, where λ_(ems) is thewavelength of signal radiation emitted from PTOLs. The PSF is especiallyimproved when high n_(sub) substrates can be used. A second SWorthogonal to the first then can be generated and scanned over the samesubset of activated PTOLs to localize them along the other axis withinthe plane.

The beams 1000 and 1001 forming a TIRF excitation field structured inthe plane of the interface also can be transmitted to the interfaceeither through a TIRF-capable signal collection objective (as shown inFIG. 10c ), or with optical elements (e.g., prisms) on the side of thesubstrate opposite the interface.

Widefield molecular localization is well suited to thin samples (toreduce out-of-focal plane fluorescence), and TIRF molecular localizationis suited to portions of the sample near the sample/substrate interface,by the evanescent field. On the other hand, confocal microscopy can beused to localize PTOLs in 3D in thick samples, such as whole cells. The3D confocal overall PSF, defined by the product of the excitation PSF(determined by focusing of the excitation) with the 3D detection PSF(defined by the confocal pinhole and the numerical aperture of thedetection objective), can be volumetrically larger than in the thinsample widefield or TIRF cases. Therefore, autofluorescence may belarger in such a case, which can reduce the localization accuracy orsuggest the use of PTOLs having higher intrinsic brightness.

However, activation and excitation energy outside the focal plane in theincoming and outgoing focal cones of the confocal microscope canprematurely activate and then photobleach PTOLs, adding to theout-of-focus background and reducing the population of PTOLs that can beaccurately localized (i.e, those near the focal plane)). Since many ofthe photoswitchable FPs (e.g., PA-GFP and Kaede) are activated by violetor near-UV light, this problem can be lessened by using multiphotonexcitation to activate the molecules, since this nonlinear processgenerally results in low PTOL activation outside the effectivemultiphoton depth of focus. The multiphoton focus could either precedethe confocal excitation focus during the latter's path through the scanvolume, or molecules across the current focal plane could first beactivated by a 2D scan of the multiphoton focus until the desireddensity of individually resolvable activated molecules is reached, to befollowed by a similar 2D scan of the confocal focus to detect andlocalize the molecules so activated. Also, when multifocal activation isused, damage to the specimen from the short wavelength of the activationbeam is likely to be greatly reduced.

Confocal molecular localization is a serial process, and thereforerelatively slow. For example, confocal molecular localization is atriply serial process because it provides activation, followed byacquisition of multiple serially scanned 3D images until all currentlyactivated molecules are bleached, followed again by activation andmultiple 3D scanning—over and over again, until a 3D map of molecularpositions of the desired density is obtained. This is obviously a slowerproposition than 2D imaging by widefield or TIRF molecular localization.Multifocal microscopy utilizing Nipkow disk technology can be used toincrease the speed of the process to some extent, but it createsmultiple foci only in a single plane, and still generates significantout-of-focal plane excitation leading to premature bleaching of targetmolecules and increased autofluorescence-induced background, even withpinhole filtering. On the other hand, 3D lattice excitation can providemany excitation maxima simultaneously in 3D, as described in PCT PatentApplication Serial No. PCT/US2005/042686 Nov. 23, 2005, and entitled“OPTICAL LATTICE MICROSCOPY,” which is incorporated herein by reference,with unintended photobleaching and associated background significantlyreduced by the improved confinement of the excitation to predominantlythese maxima alone. Furthermore, if the lattice is created withconstituent beams spread across a greater solid angle than that coveredby a single microscope objective, the confinement of the excitation ateach lattice maximum (e.g., as defined by the full volume at half peakintensity) can be greater than that in either single focus ortraditional multifocal microscopy, further reducing the backgroundsignal significantly, and permitting more accurate localization of eachPTOL, due to the tighter initial PSF. Of course, an optimal SNR andinitial PSF is expected when all beams of the maximally symmetriccomposite lattice are used. As in the confocal case, multiphotonactivation can be locally applied, such as with a multiphoton lattice,either scanned ahead of the fluorescence excitation lattice, or scannedto create a series of parallel planes of activated PTOLs prior tosimultaneous scanning of these planes by the fluorescence excitationlattice.

d. PTOL Properties

PTOLs useful for superresolution via localization of isolated PTOLsgenerally have one or more of the following distinguishingcharacteristics: a relatively high brightness (as defined by itsexcitation cross section and the quantum efficiency); a relatively highcontrast ratio between luminescence generated in the activated state tothat generated in the inactivated state (which might be improved througha judicious choice of the excitation wavelength and detection filterset); an excitation wavelength that reduces autofluorescence from othercellular material exposed to the excitation; an emission wavelength thatis sufficiently different from the spectral range over which mostautofluorescence occurs; and photostability that is large enough that asufficient number of photons are collected from each PTOL to achieve thedesired localization accuracy prior to irreversible bleaching, yet, forPTOLs other than the kindling proteins and Dronpa that can switch backto the deactivated state, is nevertheless still finite, so that a newpopulation of individually resolvable activated PTOLs can be createdafter the current set is largely bleached. Indeed, to reduce possiblephototoxicity related to irreversible photobleaching, an ideal PTOLwould remain in the activated state until it is deactivated by choiceusing other means (e.g., illumination at a separate deactivationwavelength).

Superresolution via localization has been demonstrated with thetetrameric PTOLs Kaede and Kikume, as well as the monomeric, dimeric,and tandem dimer forms of EosFP. These PTOLS have the common advantagesof large wavelength spread between the inactivated and activatedabsorption and emission maxima, high brightness, and longer wavelengthemission, where autofluorescence is typically lower. Monomeric EosFP hasthe added advantage of smaller physical size than tetrameric Kaede orKikume, and may therefore be less perturbative of cellular structure andfunction. In practice, a particular FP could be selected from a numberof different FPs based on a user's criteria for optimization for a givenapplication.

e. Background Reduction

If the contrast ratio between activated and inactivated PTOLs is too lowat a given initial density of target PTOLs to achieve the desired SNRand consequent localization accuracy, the contrast ratio can be improvedby irreversibly bleaching a portion of the target PTOLs until theeffective molecular density and resulting SNR is as desired. Otherautofluorescent material in the sample can also be pre-bleached usingthe excitation light without affecting the bulk of the inactivatedPTOLs. Further discrimination with respect to background might beobtained via appropriate spectral filtering, fluorescence lifetimemeasurements, or polarized excitation and/or polarization analyzeddetection.

In widefield microscopy, spatially structured activation energyconcentrated near the focal plane (e.g., from an axial standing wave)can be used to reduce the background from activated, out-of-focus PTOLsaway from this plane. In confocal or lattice microscopy, similarbackground reductions can be achieved with standing wave or other meansof planar, axially structured activation rather than the 3D confinedfoci traditionally applied by these methods.

f. Polarized Excitation/Detection

Light from PTOLs having an electric dipole moment located in a samplecan be detected and imaged using the techniques described herein. Whensuch PTOLs have a fixed spatial orientation in the sample, they can beselectively activated, excited, and/or imaged with polarized light. Byanalyzing the polarization of the light emitted from such PTOLs (e.g.,by passing light emitted from such PTOLs in the sample through apolarization filter prior to detecting the light such that only lighthaving a desired polarization is detected) the dipole orientations ofthese PTOLs can be determined. For a plurality of fixed dipole PTOLsthat are randomly oriented in a sample, the PTOLs can be activatedand/or excited with equal probability by using activation and/orexcitation radiation polarized in all three orthogonal directions,rather than with the unequal weightings that would result from usingactivation and/or excitation radiation having a single excitationpolarization. Because the electric field of a polarized light lies in aplane orthogonal to its direction of propagation, polarized excitationin all three directions requires at least two independent excitationbeams. For example, the through-the-objective TIRF system describedherein, inter alia, with respect to FIG. 9 is capable of delivering fourindependent beams at 90° intervals in the plane of the sample/substrateinterface, by using four excitation collimators 900, creating four inputbeams 901 that are reflected from four spots on mirror 909 and sent intothe rear pupil of objective 902. Polarizing the beams at 0° and 90°radially with respect to the rear pupil and similarly polarizing thebeams at 180° and 270° azimuthally results in two interfacial waves,polarized orthogonally with respect to one another in the plane of theinterface, and two interfacial waves polarized orthogonal to theinterface. These beams can be turned on either sequentially orsimulataneously—although in the latter case, beams having commonpolarization vectors will mutually interfere. In fact, as shown in FIG.10, by turning on the beams in pairs with like polarization, a standingwave can be formed to provide enhanced localization accuracy due to thesharp excitation PSF of the standing wave. Thus, the features ofaccurate localization, dipole determination, and equal excitationprobably of fixed dipoles can be combined.

g. Exemplary Superresolution Images

FIG. 11 compares a diffraction-limited image (FIG. 11a ) of a lysosomalstructure in a COS7 cell and superresolution image (FIG. 11b ) of thesame lysosomal structure in the same COS7 cell, which was obtained usingthe apparatus and techniques of TIRF isolation/localization describedherein. The sample containing the COS7 cell was prepared by transienttransfection with a plasmid designed for the expression of thephotoactivatable protein Kaede fused to the lysosomal transmembraneprotein CD63. Cells were pelletized and then sectioned with a microtome,using the techniques common to transmission electron microscopy, tocreate the approximately 80 nm thick section that was imaged. 20,000frames of single molecule images were taken, with activation energyapplied in a brief pulse after every 20 frames to restore the number ofactivated molecules to a higher, but still individually resolvablelevel. The superresolution image shown in FIG. 11b was formed from morethan 51,000 isolated molecules, with each molecule localized with anuncertainty of 24 nm or less redrawn in FIG. 11b as a spot having anintensity image profile given by a Gaussian distribution with a standarddeviation equal to the position uncertainty. The profiles of the spotsfor each molecule were normalized to provide the same integratedintensity for each molecule. Thus, more highly localized moleculesappear as bright, sharp dots, and less well localized ones appear broadand dim. The diffraction limited image was formed by summing thediffraction limited images of the same set of isolated molecules, andwas verified to be indistinguishable from the conventional TIRF image.

FIG. 12 compares a diffraction-limited image (FIG. 12a ) obtained at theinterface of a whole, fixed fox lung fibroblast cell and a glass coverslip in phosphate buffered saline and a superresolution (FIG. 12b )image of the same fox lung fibroblast cell. The cell was transientlytransfected to express the photoactivatable protein dEosFP fused to thecell attachment protein vinculin. The images were created in the samemanner as described in conjunction with FIG. 11. The diffraction limitedimage highlights a single focal adhesion region at the periphery of thecell, and the superresolution image by PTOL localization shows amagnified view of the structure within the box in FIG. 12 a.

3. Enhanced Resolution Via Overlapped Spatially Structured Activationand Excitation

The overall PSF of any form of optical microscopy (e.g., widefield,TIRF, confocal, or lattice) is typically given by the product of theexcitation PSF with that of the detection PSF (i.e.,PSF_(overall)=PSF_(excitation)×PSF_(detection)). Widefield microscopyoffers no excitation contribution to the resolution, traditional TIRFmicroscopy offers very high z-axis excitation resolution, but none inthe x- and y-axes, and both confocal and lattice microscopy contributeexcitation resolution by concentration of the excitation field to eithera single focus, or a lattice of intensity maxima.

PTOLs offer a way of contributing a third component to the overall PSFby confining the activation illumination to a localized region in amanner similar to that used to confine the excitation energy itself(i.e., PSF_(overall)=PSF_(activation)×PSF_(excitation)×PSF_(detection))Thus, for example, a focused activation beam can be temporarily appliedat the focal point of a confocal microscope, followed by a focused beamat the excitation wavelength for the activated PTOLs, with the resultingemission being detected confocally in a spatially localized manner. Thisprocess can then be repeated over many voxels (i.e., a 3D pixel) tocreate a complete superresolution 3D image. One caveat is that thenumber of activated PTOLs in a focal volume should decline significantly(either by irreversible photobleaching, or reversion to the unactivatedstate) before activation and excitation is applied to an immediatelyneighboring voxel, or else the effective activation PSF will bereflective of the larger region defined by the overlapping, neighboringactivation foci, thereby degrading the effective overall PSF. Dronpaappears to be a particularly good candidate for this method ofsuperresolution, because the activated molecules are returned to thedeactivated state by the process of their excitation, thereby providinga natural means to depopulate the activated ensemble whilesimultaneously determining when the scan should proceed to the nextvoxel. If the deactivation occurs too quickly, multipleactivation/(deactivation and measurement) cycles can be performed at thesame position before proceeding to the next position. Using Dronpa asthe PTOL in this process allows more than about 100 such cycles to beperformed at each position.

Because the activation wavelength is typically short (e.g., about 400nm) for Dronpa, the activation PSF can provide most of the resolutionbenefit in the overall PSF. If cellular damage from this short of awavelength is a concern, multiphoton activation can be used, at the costof a slightly larger activation PSF than is possible with linear (i.e.,single photon) activation. In addition, because the density of emittingmolecules is given by PSF_(activation)×PSF_(excitation) the emittingmolecules will be confined to at least as tight a focal region as inconventional two-photon excitation, thereby leading to greatly reducedout-of-plane photobleaching and background, even using linear, confocalexcitation. Of course, further gains in both spatial and temporalresolution are possible if sparse composite lattices of the same orcommensurate periods are used for both the activation radiation (asshown in FIG. 13a , and in the close up view of FIG. 13a shown in FIG.13d ) and for the excitation radiation (as shown in FIG. 13b , and inthe close up of FIG. 13b in FIG. 13e ), leading to an overall lattice(shown in FIG. 13c ) that achieves activation and excitation of PTOLsand that has having sharper maxima (shown in FIG. 13f ) than the maximain the lattices for the activation and excitation radiation. Pointspread function engineering and relative displacement of the activationand excitation PSFs might be used to further increase the resolution byreducing the region of their effective overlap. If the contribution ofthe detection PSF to the overall resolution is negligible, it might beadvantageous to simply omit pinhole filtering (as in most embodiments ofmultiphoton microscopy) in order to maximize the collected signal.Finally, we note that this method of superresolution, is well suited todynamic superresolution imaging in living cells (particularly withlattice microscopy), because potentially many more molecules would beemitting photons at a given time from each focus (when confocalradiation is used for activation and excitation) or excitation maximum(when radiation patterned in a lattice is used for activation andexcitation).

4. Superresolution Via Saturated Deactivation

By exploiting the saturation of the deactivation of PTOLs over asub-portion of a diffraction-limited focal volume in which a portion ofthe PTOLs were previously activated, one can collect emission from asub-diffraction limited region, and then repeat at multiple locations togenerate a sub-diffraction limited image. This concept is described inFIG. 14 in reference to activation, deactivation, and excitation withoptical lattices, although other means (e.g., single focused beams) canalso be employed.

As shown in FIG. 14a , a lattice of confined intensity maxima can befirst created at the activation wavelength of the PTOLs to create anarray of localized regions of activated PTOLs. Next, a depletion lattice(as shown in FIG. 14b ) having a central low intensity node within ashell of high intensity located at each lattice point, can be applied ata wavelength that returns the PTOLs outside each node to theirunactivated state. Next, an excitation lattice (as shown in FIG. 14c )can be applied at the excitation wavelength of the activated PTOLs, sothat the small (e.g., having dimensions that can be less than thewavelength of the emission radiation) volume of PTOLs near each node ofthe depletion lattice is excited and then emits photons, resulting inthe desired lattice of superresolution foci (as shown in FIG. 14d ).Next, the remaining activated PTOLs are deactivated, such as by excitingthem until a substantial fraction of them photobleach, or by applying adeactivation radiation until a substantial fraction of them are returnedto the unactivated.state. This process of activation, partialdeactivation with a nodal pattern, excitation, and nearly completedeactivation can then be repeated at different points to create alattice of superresolution foci offset from the first. By repeating theprocess further at a multiplicity of points across each primitive cellof the lattice, and detecting emission radiation from individualsuperresolution foci in a given cycle of activation/nodaldeactivation/excitation/complete deactivation at separate detectionelements (e.g., the pixels of a CCD detector), a complete 3D image canbe constructed (as shown in FIG. 14f ), at considerably higherresolution than is possible, for example, by conventional confocalmicroscopy (as shown in FIG. 14e ). All three lattices (i.e., thelattices of the activation radiation, the depletion radiation, and theexcitation radiation) can be chosen at wavelength-normalizedperiodicities, such that the ratios of their absolute periodicities formsimple integer fractions (i.e., i/j), or ideally, have the same absoluteperiodicity (i/j=1), so that many of the activation maxima, deactivationdepletion shells, and excitation maxima overlap. The completelydeactivating radiation can also be applied in the form of a lattice, oras a substantially uniform deactivation field.

Again considering more general radiation patterns, it is important tonote that only the nodal deactivation radiation pattern needs to bespatially structured (specifically, with at least one low intensitynode), and that even uniform activation, excitation, or completedeactivation radiation fields may be applied. However, it may bebeneficial to spatially structure either or both of the activation andexcitation fields as well, in order to increase the contrast between thedesired remaining activated PTOLs near the nodes after deactivationrelative to undesired remaining activated PTOLs elsewhere, and to reducethe potential damage to reversible PTOLs by repeated activation anddeactivation cycles. More thorough final deactivation near the nodes mayalso be attained by spatially structuring the complete deactivationradiation as well, to concentrate it at these points of residualactivation. Also note that the density and spatial confinement of theactivated PTOLs remaining after application of the deactivation energyis improved if the deactivation field is closer to zero intensity at thenodes, and if the rate of decrease in deactivation intensity near thenodes is high.

Specific photoactivatable FPs can be used in this technique. Forexample, kindling proteins, such as KFP1 and dronpa, can be used becausethey both can be photoswitched back to an unactivated state. KFP1requires low intensity activation to insure that the molecules are notirreversibly activated, has a relatively low quantum efficiency, anddeactivation of KFP1 occurs at a different wavelength than theexcitation. Dronpa exhibits high brightness and is demonstrablyswitchable over many cycles, but time gating of the detection signal isrequired, because the depletion wavelength is the same as the excitationwavelength, so the fluorescence generated during depletion of theactivated state must be rejected, or collected separately from the lateremission near the depletion nodes. On the other hand, the emissioncollected during depletion can be used to generate a high SNRdiffraction-limited image, since many more molecules would contribute tothe emission during depletion.

5. PTOL Imaging at Reduced Temperatures

Biological samples labeled with PTOLs that are alive or at roomtemperature pose special challenges for this localization microscopy.The PTOL labels might diffuse or be transported beyond the localizationaccuracy during the relatively long multi-frame acquisition time. Inaddition, other properties such as the PTOL orientation could be varyingduring the acquisition resulting in the loss of potentially usefulmicroscopic information. Thus, cooling a sample below room temperaturecan reduce the movement of a sample while it is imaged.

In addition, at reduced temperatures, the brightness and spectral linewidths of certain PTOLs improve, so that more photons can be acquiredmore quickly for better resolved localization images, and contrast ofthe PTOLs relative to autofluorescence background may be reduced.Included here is an implementation where the sample or the sample andparts of the microscope is cooled below freezing temperatures tomitigate these limitations. In particular, rapid freezing can preparesamples in a vitreous state so that no potentially damaging ice crystalsare formed within the sample

6. PTOL Microscopy of Latent Images

In lithography, nanometer scale patterns can be written with photon,electron, ion or atom beams. Typically the pattern is written onto abeam sensitive material such as a resist. In the cases of optical orelectron beams, photoresist or e-beam resists can be used. For optimallithographic performance it is useful to characterize the precise shapeof the beam and the exposed pattern at an early stage before subsequentprocessing transfers the exposed resist pattern onto other materials.Thus, a resist can contain PTOLs or be labeled with PTOLs on the top orbottom surfaces of the resist layer. In this case, contrast can beimposed by the exposing beam by several kinds of exposure beams, and theexposure beam can have a detectable effect on the PTOLs in the resist,such that imaging the PTOLs after exposure can reveal the pattern of theexposure beam in the resist. For example, such an exposure beam can:destroy a PTOLs ability to radiate (e.g. by electron beam ionization, orUV induced bond breaking, etc.); shift the emission wavelength of thePTOL (e.g., in a manner similar to the wavelength shift in Kaede due toactivation radiation); or catalyze the release of an acid in the resist,as is common in the case of chemically activated resists, which thenchanges the photophysical properties of the exposed PTOLs. Thus, asshown in FIG. 15a , to resist can include a number of PTOLs. As shown inFIG. 15b , when a portion 1502 of the resist is exposed to exposureradiation in a lithography process, photo-lithographically activatedacids 1503 can catalyze further cleavage or polymerization of resist. Inaddition, the acids 1503 can also shift the emission wavelength of thePTOLs (e.g., when Eos is used as the PTOL 1501), where an activatedstate of the PTOL 1506 in the presence of the acid 1503 can emit morestrongly at a different wavelength than when the PTOL 1501 is not in thepresence of the acid 1503. A PTOL microscope could image the acidtransformed PTOLs 1506 or the nontransformed PTOLs 1501 as a latentimage on the resist. This in turn could provide a measure of theexposure properties and profiles at the resolution of the PTOLlocalization length scale.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. All publications, patentapplications, patents, and other references mentioned herein areincorporated by reference in their entirety. In case of conflict, thepresent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and notlimiting.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A method comprising: providing activationradiation to a sample, wherein the sample includes at least a firstspecies of phototransformable optical labels (“PTOLs”) that label firstmolecules in the sample and a second species of PTOLs that label secondmolecules in the sample, wherein the second molecules are different fromthe first molecules, the first and second species of PTOLs beingdistributed in the sample such that separation distances between atleast some first species of PTOLs and second species of PTOLs are lessthan a diffraction limited resolution of a detection objective ofimaging optics used to image radiation from the PTOLs, to activate asubset of the PTOLs in the sample; providing excitation radiation to thesample to excite at least some of the activated subset of PTOLs;detecting, with the imaging optics, radiation emitted from the activatedand excited PTOLs within the subset of PTOLs, the detected radiationincluding radiation from the first and second species of PTOLs;controlling the activation radiation such that adjacent activated PTOLsin a focal plane of the detection objective of the imaging optics areseparated by distances that are, on average, larger than the diffractionlimited resolution of the detection objective; distinguishing the firstspecies of PTOLs from the second species PTOLs based on at least one of:emission characteristics of the first and second species and excitationcharacteristics of the first and second species; and determininglocations of the first and second species of the PTOLs based on thedetected radiation.
 2. The method of claim 1, wherein the first andsecond molecules include proteins.
 3. The method of claim 1, furthercomprising determining a co-localization of first and second moleculeson targets in the sample based on the determined locations of the firstand second species of the PTOLs.
 4. The method of claim 1, wherein firstand second molecules bind to each other to form a molecular complex inthe sample.
 5. The method of claim 1, wherein first and second moleculesbind to targets in the sample and the method further comprisingdetermining a spatial orientation of the first molecules bound to thetargets relative to the second molecules bound to the targets based onthe detected radiation.
 6. The method of claim 1, wherein first andsecond molecules bind to targets in the sample and further comprisingdetermining a binding affinity of the first molecules to the targetsrelative to a binding affinity of the second molecules to the targets.7. The method of claim 1, wherein first and second molecules areattached to targets in the sample and further comprising determining adistance between the first molecules and the second molecules on thetargets.
 8. The method of claim 1, wherein the activation radiation hasan activation wavelength and wherein the excitation radiation has anexcitation wavelength that is longer than the activation wavelength. 9.The method of claim 1, wherein the imaging optics include an objectivelens, wherein detecting radiation emitted from the activated and excitedPTOLs includes detecting the emitted radiation with a position-sensitivedetector, and wherein a magnification of the imaging optics and sizes ofpixels in the position-sensitive detector is selected such that imagesof PTOLs fall on multiple pixels of the position-sensitive detector. 10.The method of claim 1, further comprising modulating the activationradiation that is provided to the sample to activate different subsetsof activated PTOLs.
 11. The method of claim 1, wherein the first andsecond species of PTOLs include fluorescent proteins.
 12. The method ofclaim 1, wherein providing activation radiation to the sample includesproviding the activation radiation to the sample to activate a pluralityof different subsets of the PTOLs, wherein the different subsets of theoptical labels are activated at different times, wherein providing theexcitation radiation to the sample includes providing the excitationradiation to the sample to excite at least some of the PTOLs in thedifferent activated subsets, wherein controlling the activationradiation includes controlling the activation radiation such thatadjacent activated PTOLs in each of the different subsets are separatedby distances that are, on average, larger than the diffraction limitedresolution of the detection objective, wherein detecting the radiationincludes detecting radiation emitted from the activated and excitedPTOLs within the different subset of PTOLs, wherein radiation from thePTOLs within the different subsets is detected at different times, andwherein determining locations of the PTOLs includes determininglocations of PTOLs in different ones of the subsets based on thedetected radiation.
 13. The method of claim 1, wherein the radiationfrom the first and second species of PTOLs is detected at differenttimes.
 14. The method of claim 1, wherein the sample includes aplurality of different molecules from a library of molecules, differentmolecules of the library being labeled by different species ofphototransformable optical labels (“PTOLs”), the PTOLs being distributedin the sample such that separation distances between PTOLs are less thana diffraction limited resolution of a detection objective of imagingoptics used to image radiation from the PTOLs and further comprising:detecting, with the imaging optics, radiation emitted from the activatedand excited PTOLs within the subset of PTOLs, the detected radiationincluding radiation from a plurality of species of PTOLs that labeldifferent molecules from the library; controlling the activationradiation such that adjacent activated PTOLs in a focal plane of thedetection objective of the imaging optics are separated by distancesthat are, on average, larger than the diffraction limited resolution ofthe detection objective; determining locations of the PTOLs, includingthe different species of PTOLs, based on the detected radiation; anddistinguishing different species of PTOLs based on at least one of:emission characteristics of the different species and excitationcharacteristics of the different species; and determining a bindingaffinity of a molecule composed of a plurality of different moleculesfrom the library with a target in the sample based on the determinedlocations of the PTOLs.
 15. A method comprising: (a) providingactivation radiation to a sample, wherein the sample includesphototransformable optical labels (“PTOLs”) distributed in the samplesuch that separation distances between PTOLs are less than a diffractionlimited resolution of a detection objective of imaging optics used toimage radiation from the PTOLs, to activate a subset of the PTOLs in thesample; (b) controlling the activation radiation such that adjacentactivated PTOLs in a focal plane of the detection objective of theimaging optics are separated by distances that are, on average, largerthan the diffraction limited resolution of the detection objective; (c)providing excitation radiation to the sample to excite at least some ofthe activated subset of PTOLs; (d) detecting, with the imaging optics,radiation emitted from an activated and excited PTOL bound to a proteinwithin the subset of PTOLs; (e) repeating (c) and (d) at differenttimes; and (f) determining locations of the activated and excited PTOLbound to the protein at the different times based on the detectedradiation.
 16. The method of claim 15, wherein the protein binds to atarget in the sample at least at some of the different times and themethod further comprising determining a spatial orientation of theprotein when it is attached to the target based on the detectedradiation.
 17. The method of claim 15, wherein the protein binds to atarget in the sample and the method further comprising determining abinding affinity of the protein to the target based on the detectedradiation.
 18. The method of claim 15, wherein the activation radiationhas an activation wavelength and wherein the excitation radiation has anexcitation wavelength that is longer than the activation wavelength. 19.The method of claim 15, wherein the imaging optics include an objectivelens, wherein detecting radiation emitted from the activated and excitedPTOL bound to the protein includes detecting the emitted radiation witha position-sensitive detector, and wherein a magnification of theimaging optics and sizes of pixels in the position-sensitive detector isselected such that an image of the PTOL falls on multiple pixels of theposition-sensitive detector.
 20. The method of claim 15, wherein thesample includes a molecule selected from a library of differentmolecules.
 21. The method of claim 20, wherein molecules of the libraryof different molecules include drug fragments.